Automatic Generation of Test Oracles

We describe a progression fiom pilot studies to development and use of domain-spec fzc verification and validation (V& v ) automation. Our domain is the testing of an AIplanning system that forms a key component of an autonomous spacecraft. We used pilot studies to ascertain opportunities for, and suitability OJ; automating various analyses whose results would contribute to V& V in our domain. These studies culminated in development of an automatic generator of automated test oracles. This was then applied and extended in the course of testing the spacecraft’s AIplanning system. (Richardson et al, 1992) presents motivation for automatic test oracles, and considered the issues and approaches particular to test oracles derivedfiom specfzcations. Our work, carried through from conception to application, confirms many of their insights. Generalizing @om our speclfic domain, we present some additional insights and recommendations concerning the use of test oracles for V& V of knowledge-based systems.